منابع مشابه
Potentials for Denumerable Markov Chains
Probabilistic generalizations of classical potential theory have been worked out by J. L. Doob [ l ] and G. A. Hunt [2]. Specialized to discrete processes, this provides a potential theory for transient Markov chains, analogous to the theory of the Newtonian potential. We provide a unified potential theory for all denumerable Markov chains; as applied to recurrent chains, the theory generalizes...
متن کاملOn Weak Lumpability of Denumerable Markov Chains
We consider weak lumpability of denumerable discrete or continuous time Markov chains. Firstly, we are concerned with irreducible recurrent positive and R-positive Markov chains evolving in discrete time. We study the properties of the set of all initial distributions of the starting chain leading to an aggregated homogeneous Markov chain with respect to a partition of the state space. In parti...
متن کاملA note on exponential stability of the nonlinear filter for denumerable Markov chains
We study asymptotic stability of the optimal filter with respect to its initial conditions. We show that exponential stability of the nonlinear filter holds for a large class of denumerable Markov chains, including all finite Markov chains, under the assumption that the observation function is one-to-one and the observation noise is sufficiently small. Ergodicity of the signal process is not as...
متن کاملDenumerable Constrained Markov Decision Problems and Finite Approximations Denumerable Constrained Markov Decision Problems and Finite Approximations
The purpose of this paper is two fold. First to establish the Theory of discounted constrained Markov Decision Processes with a countable state and action spaces with general multi-chain structure. Second, to introduce nite approximation methods. We deene the occupation measures and obtain properties of the set of all achievable occupation measures under the diierent admissible policies. We est...
متن کاملPerturbation Analysis for Denumerable Markov Chains with Application to Queueing Models
We study the parametric perturbation of Markov chains with denumerable state spaces. We consider both regular and singular perturbations. By the latter wemean that transition probabilities of aMarkov chain, with several ergodic classes, are perturbed such that (rare) transitions among the different ergodic classes of the unperturbed chain are allowed. Singularly perturbed Markov chains have bee...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Bulletin of the American Mathematical Society
سال: 1961
ISSN: 0002-9904
DOI: 10.1090/s0002-9904-1961-10532-6